Asymptotic law of likelihood ratio for multilayer perceptron models

We consider regression models involving multilayer perceptrons (MLP) with one hidden layer and a Gaussian noise. The data are assumed to be generated by a true MLP model and the estimation of the parameters of the MLP is done by maximizing the likelihood of the model. When the number of hidden units of the true model is known, the asymptotic distribution of the maximum likelihood estimator (MLE) and the likelihood ratio (LR) statistic is easy to compute and converge to a $\chi^2$ law. However, if the number of hidden unit is over-estimated the Fischer information matrix of the model is singular and the asymptotic behavior of the MLE is unknown. This paper deals with this case, and gives the exact asymptotic law of the LR statistics. Namely, if the parameters of the MLP lie in a suitable compact set, we show that the LR statistics is the supremum of the square of a Gaussian process indexed by a class of limit score functions.Comment: 19 page

Estimating the Number of Regimes of Non-linear Autoregressive Models.

International audienceAutoregressive regime-switching models are being widely used in modelling financial and economic time series such as business cycles (Hamilton, 1989; Lam, 1990), exchange rates (Engle and Hamilton, 1990), financial panics (Schwert, 1989) or stock prices (Wong and Li, 2000). When the number of regimes is fixed the statistical inference is relatively straightforward and the asymptotic properties of the estimates may be established (Francq and Roussignol, 1998; Krishnamurthy and Rydén, 1998; Douc R., Moulines E. and Rydén T., 2004). However, the problem of selecting the number of regimes is far less obvious and hasn't been completely answered yet. When the number of regimes is unknown, identifiability problems arise and the likelihood ratio test statistic (LRTS hereafter) is no longer convergent to a $\chi^{2}$-distribution. In this paper, we consider models which allow the series to switch between regimes and we propose to study such models without knowing the form of the density of the noise. The problem we address here is how to select the number of components or number of regimes. One possible method to answer this problem is to consider penalized criteria. The consistency of a modified BIC criterion was recently proven in the framework of likelihood criterion for linear switching models (see Oltéanu and Rynkiewicz). We extend these results to mixtures of nonlinear autoregressive models with mean square error criterion and prove the consistency of a penalized estimate for the number of components under some regularity conditions

Self Organizing Map algorithm and distortion measure

International audienceWe study the statistical meaning of the minimization of distortion measure and the relation between the equilibrium points of the SOM algorithm and the minima of distortion measure. If we assume that the observations and the map lie in an compact Euclidean space, we prove the strong consistency of the map which almost minimizes the empirical distortion. Moreover, after calculating the derivatives of the theoretical distortion measure, we show that the points minimizing this measure and the equilibria of the Kohonen map do not match in general. We illustrate, with a simple example, how this occurs

Asymptotics for regression models under loss of identifiability

This paper discusses the asymptotic behavior of regression models under general conditions. First, we give a general inequality for the difference of the sum of square errors (SSE) of the estimated regression model and the SSE of the theoretical best regression function in our model. A set of generalized derivative functions is a key tool in deriving such inequality. Under suitable Donsker condition for this set, we give the asymptotic distribution for the difference of SSE. We show how to get this Donsker property for parametric models even if the parameters characterizing the best regression function are not unique. This result is applied to neural networks regression models with redundant hidden units when loss of identifiability occurs

Testing the number of parameters with multidimensional MLP

International audienceThis work concerns testing the number of parameters in one hidden layer multilayer perceptron (MLP). For this purpose we assume that we have identifiable models, up to a finite group of transformations on the weights, this is for example the case when the number of hidden units is know. In this framework, we show that we get a simple asymptotic distribution, if we use the logarithm of the determinant of the empirical error covariance matrix as cost function

Estimation of linear autoregressive models with Markov-switching, the E.M. algorithm revisited.

International audienceThis work concerns estimation of linear autoregressive models with Markov-switching using expectation maximisation (E.M.) algorithm. Our method generalise the method introduced by Elliot for general hidden Markov models and avoid to use backward recursion

Consistance d'un estimateur de minimum de variance étendue

National audienceWe consider a generalization of the criterion minimized by the K-means algorithm, where a neighborhood structure is used in the calculus of the variance. Such tool is used, for example with Kohonen maps, to measure the quality of the quantification preserving the neighborhood relationships. If we assume that the parameter vector is in a compact Euclidean space and all it components are separated by a minimal distance, we show the strong consistency of the set of parameters almost realizing the minimum of the empirical extended variance

Estimation consistante de l'architecture des perceptrons multicouches

National audienceWe consider regression models involving multilayer perceptrons (MLP) with one hidden layer and a Gaussian noise. The estimation of the parameters of the MLP can be done by maximizing the likelihood of the model. In this framework, it is difficult to determine the true number of hidden units because the information matrix of Fisher is not invertible if this number is overestimated. However, if the parameters of the MLP are in a compact set, we prove that the minimization of a suitable information criteria leads to consistent estimation of the true number of hidden units